Camera devices and mobile phone manufacturers are demanding increased image sensor quality and higher image resolution, while keeping the same physical dimension of the device and the image sensor. As a result, the image sensor uses smaller pixel sizes to enable higher resolutions. However smaller pixel sizes tend to provide a lower signal to noise ratio (SNR) especially in low light conditions and when using digital zoom. Accordingly, it is desirable to find methods to improve image quality.
When taking pictures in low-light conditions, a conventional camera increases exposure using some or all of the following methods: (a) increased shutter time, (b) increased aperture, (c) increased ISO number.
(a) Increasing the shutter time directly increases the amount of light, however this increases the opportunity for motion blur, i.e. the resulting picture will blur if the camera or the subject moves during the increased exposure time.
(b) An increased aperture directly increases the amount of light in the exposure, but also decreases the depth of field. Additionally there is a limit on the maximum aperture for a specific lens (smallest f-stop number) limiting the aperture size. Furthermore, in some camera equipment such as in standard smart phones nowadays the aperture size is fixed.
(c) Increase ISO number which control sensitivity of digital imaging systems. Unfortunately, the higher the sensitivity, the grainier the images become and the amount of noise increases. A higher ISO number produces noisier images due to higher sensor amplification and lower initial SNR defined by the ISO number.
The following types of conventional anti-shake/anti-blur technologies are currently implemented in low-light photography: 1. mechanical anti-shake and 2. digital anti-shake/anti-blur. The mechanical anti-shake technology moves the lens or sensor while the shutter is open to counteract camera motion (global motion). The conventional digital anti-shake/anti-blur technology increases the ISO number while the shutter time is correspondingly reduced. This reduces blur at the expense of granularity and image noise. Mechanical anti-shake produces much better image quality (higher SNR) as the shutter can remain open longer and the temporal picture integration reduces noise. However, the mechanical anti-shake implementations require additional physical components adding to the overall expense of the camera, and tends to provide ghost images and blur in case of local motion within the scene being recorded. It is desirable to employ a digital technique that can achieve a better tradeoff between image noise and motion blur than the conventional technique listed above.
Nowadays, many digital cameras are able to take multiple pictures at once in a “burst mode”, for example Samsung Galaxy S3 includes a camera that can take up to 20 pictures each having 8 Megapixel within 6 sec (3.3 fps). This feature is commonly used to track events/objects/features on a frame by frame basis such as a person's smile or eyes. This technology may be leveraged to improve SNR by allowing the user to manually choose a desired picture from the multiple pictures, or automatically select a specific picture.
Reducing image noise for example noise resulting from dark current, photon noise, and cross-talk can improve the resulting images. The signal to noise ratio (SNR) may be particularly low for low light photography and would be greatly affected by such a reduction. One way to reduce noise in an image is to run an edge adaptive spatial low pass filter over an image while using an edge detector to protect some of the edge boundaries. However, even if some of the edges are protected, the filter affects the textures in the scene, because it may be difficult to discern between texture and noise.
Another way to improve SNR i.e. reduce noise is by temporally combining matching parts from two or more images by applying a temporal filter (e.g. a Motion Compensated Temporal Filtering (MCTF)). By temporally combining the spatial filter problems described above can be avoided however other artifacts such as ghosting or blur may be introduced and the process is computationally expensive.
HDR (High-dynamic-range) imaging is a set of methods used in imaging and photography to capture a greater dynamic range between the lightest and darkest areas of an image in contrast to standard digital imaging methods or photographic methods. HDR images can represent more accurately the range of intensity levels found in real scenes, from direct sunlight to faint starlight, and the images are often captured by exposing the same subject matter with a plurality of different exposure settings or levels.
HDR methods provide a higher dynamic range from the imaging process. Non-HDR cameras take pictures at one exposure level with a limited contrast range. This results in the loss of detail in bright or dark areas of the picture, depending on whether the camera had a low or high exposure setting. HDR compensates for this loss of detail by taking multiple pictures at different exposure levels and intelligently stitching them together to produce a picture that is representative in both dark and bright areas.
HDR is also commonly used to refer to the display of images derived from HDR imaging in a way that exaggerates contrast for artistic effects. The two main sources of HDR images are computer renderings and merging of multiple low-dynamic-range (LDR) photographs or standard-dynamic-range (SDR) photographs. Tone mapping methods, which reduce overall contrast to facilitate display of HDR images on devices with lower dynamic range, can be applied to produce images with preserved or exaggerated local contrast for artistic effect. High-dynamic-range photographs are generally achieved by capturing multiple standard photographs, often using two or three different exposures, and then merging them into an HDR image.
Scenes with high dynamic ranges are often represented on LDR devices by cropping the dynamic range, cutting off the darkest and brightest details, or alternatively with an S-shaped conversion curve that compresses contrast progressively and more aggressively in the highlights and shadows while leaving the middle portions of the contrast range relatively unaffected.
Tone mapping reduces the dynamic range, or contrast ratio, of the entire image, while retaining localized contrast (between neighboring pixels), tapping into research on how the human eye and visual cortex perceive a scene, trying to represent the whole dynamic range while retaining realistic color and contrast.